Robust Processing of Real-World Natural-Language Texts

I1. is often assumed that when natural language processing meets the real world, the ideal of aiming for complete and correct interpretations has to be abandoned. However, our experience with TACITUS; especially in the MUC-3 evaluation, has shown that. principled techniques fox' syntactic and pragmatic analysis can be bolstered with methods for achieving robustness. We describe three techniques for making syntactic analysis more robust -an agendabased scheduling parser, a recovery technique for failed parses, and a new technique called terminal substring parsing. For pragmatics processing, we describe how the method of abductive inference is inherently robust, in that an interpretation is always possible, so that in the absence of the required world knowledge, performance degrades gracefully. Each of these techlfiques have been evaluated and the results of the evaluations are presented. 1 I n t r o d u c t i o n If automat ic text processing is to be a useful enterprise, it. must be demonstrated that the completeness and accuracy of the information extracted is adequate for the application one has in nfind. While it is clear that certain applications require only a minimal level of competence from a system, it is also true that many applica t ionsrequire a very high degree of completeness and a.ccuracy in text processing, and an increase in capability in either area is a clear advantage. Therefore we adopt an extremely lfigh standard against which the performance of a text processing system should be measured: it. should recover all information that is implicitly or explicitly present in the text, and it should do so without making mistakes. Tiffs standard is far beyond the state of the art. It is an impossibly high standard for human beings, let alone machines. However, progress toward adequate text processing is best. served by setting ambitious goals. For this reason we believe that, while it may be necessary in the intermediate term to settle for results that are far short of this ult imate goal, any linguistic theory or system architecture that is adopted should not be demonstrably inconsistent with attaining this objective. However, if one is interested, as we are, in the potentially successful application of these intermediateterm systems to real problems, it is impossible to ignore the question of whether they can be made efficient enough and robust enough for application. 1.1 T h e T A C I T U S S y s t e m The TACITUS text processing system has been under development at SRI International for the last six years. This system has been designed as a first step toward the realization of a system with very high completeness and accuracy in its ability to extract information from text. The general philosophy underlying the design oI this system is that the system, to the max imu m extent possible, should not discard any information that might be semantically or pragmatically relevant to a full, correct interpretation. The effect of this design philosophy on the system architecture is manifested in the following characteristics: * TACITUS relies on a large, comprehensive lexicon containing detailed syntactic subcategorization information for each lexieal item. . TACITUS produces a parse and semantic interpretation of each sentence using a comprehensive grammar of English in which different possible predicateargument relations are associated with different syntactic structures. • TACITUS relies on a general abductive reasoning meclmnism to uncover the implicit assumptions necessary to explain the coherence of the explicit text. These basic design decisions do not by themselves distinguish TACITUS from a number of other naturallanguage processing systems. However, they are somewhat controversial given the intermediate goal of producing systems that are useful for existing appl icat ions Criticism of the overall design with respect to this goal centers on the following observations: • The syntactic structure of English is very complex and no g rammar of English has been constructed that has complete coverage of the syntax one encounters in real-world texts. Much of the text thai needs to be processed will lie outside the scope ol the best g rammars available, and therefore canno! be understood by a. system that relies on a complet(